Lack of confidence in ABC model choice

Approximate Bayesian computation (ABC) have become an essential tool for the analysis of complex stochastic models. Grelaud et al. (2009, Bayesian Ana 3:427–442) advocated the use of ABC for model choice in the specific case of Gibbs random fields, relying on a inter-model sufficiency property to show that the approximation was legitimate. We implemented ABC model choice in a wide range of phylogenetic models in the DIY-ABC software (Cornuet et al. (2008) Bioinfo 24:2713–2719). We now present arguments as to why the theoretical arguments for ABC model choice are missing, since the algorithm involves an unknown loss of information induced by the use of insufficient summary statistics. The approximation error of the posterior probabilities of the models under comparison may thus be unrelated with the computational effort spent in running an ABC algorithm. We then conclude that additional empirical verifications of the performances of the ABC procedure as those available in DIYABC are necessary to conduct model choice. 1